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ABSTRACT 

This paper explores the issue of "state policy" and 
the assessment process and how it can be applied to improving the 
teaching and learning process in undergraduate education. Three 
distinct policy questions are addressed: (1) how can assessment focus 
attention on what college graduates in a democratic society should 
know and be able to do? (2) what assessment information is useful for 
making sure that the curriculum provides the necessary skills and 
content? and (3) how can assessment support departments and 
disciplines in assuming collective responsibility for the outcomes of 
college curricula? Next, assessment and its impact on Improving 
teaching is discussed. This discussion asks how assessment can help 
to inform the teaching and learning process within the classroom, how 
assessment can be used in designing state policies that support good 
college teaching, and how assessment can be linked to the 
institutional reward structure to support good teaching. Finally, the 
paper examines resource allocation in the assessment process in order 
to make assessments viable policy tools for imurr>vem«nt . This 
discussion asks how assessment can be related to the determination of 
resource needs, how assessment can help to inform the allocation of 
resources, and how assessment can be used to examine and increase the 
"return on investment" of public resources in higher education. 
(GLR) 
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INTRODUCTION 



The rhetoric of assessment in higher education, regardless of its source, has always been 
the improvement of teaching and learning. For those concerned with assessment within the 
academy, this motive is a given. Absent a visible connection to teaching and learning, 
faculty argue, there is littie point to the exercise. For the most part, state policy maicers 
agree. Radier than narrow "accountability,** they see in assessment a way to redirect higher 
education's attention toward critical societal challenges for the future. Atowe all, they see 
in assessment a means to induce faculty at colleges and universities to takt active 
responsibilitv for student learning, an obligation which many feel, over the years, has been 
badly neglected 

How to actually use state policy to achieve these ends, however, remains elusive. Partly 
tills is because the entities to be linked "state policy" and tiie "improvement of teaching 
and learning" are themselves ill specified ParUy it is because the causal chain between 
what a legislature enacts and what a classroom teacher does is dauntingly complex. We 
intend tiiis seminar to explore botii muricy areas in a manner tiiat botii can promote future 
dialogue and tiiat can provide some immediate guidance to those at tiie state level who will 
provide the immediate policy environment within which we all must function. What 
follows are brief discussions of tiuee related policy areas. Witiiin each is a series of 
questions that we are asking aU participants to address, and tiiat we hope will provide a 
framework for our deliberations. Like tiiose posed in assessment itself, we do not expect 
these questions (or tiieir answers) to be defmitive; tiiey have done tiieir job if only our 
tiiinking is clarified 
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ASSESSMENT AND THE CURRICULUM 



Assessment and undergraduate curricular reform are intimately related, for the language uf 
assessment itself implies a curricular agenda. "Assessment" Hrst requires some explicit 
standards of perfomiance; curricula that contain none cannot bo assessed. More 
importantly, '^assessment" implies an instructional design that is both coherent and 
interconnected. Even oosine assessment as a required activity will inevitably raise 
important curricolar issues. As a result, both institudonal and state proponents of 
assessment are apt particularly to emphasize its role in promoting curriculai change; indeed, 
curricular change in itself is often seen as convincing evidence tiiat the process is 
"working." To inform assessment's impact on the curriculum, we believe, three distinct 
policy questions must be addressed: 

1. How can assessment focus attention on what we expect college graduates in a 
democratic society to know and be able to do ? 

Though firequentiy seen as a prerequisite for assessment, the development of more 
explicit goals for teaching has proven to be one of its most ubiquitous actual 
products. And policy makers repeatedly claim tiiat clear goals are badly needed. 
Part of the problem here is that most institutions cannot as yet provide answers to 
the question that policy makers most want to know: what is the outcome of tiie 
curriculum as a whole ? This question is important, because policy makers do no: 
see undergraduate education as an end in itself - a "public service" to be 
provided - but rather as a critical investment in the future. But many issues spring 
from this simple demand 

First, what ought curricular goals to look like ? Is it meaningful, for example, to 
formulate tiiis question for an entire state, such as New Jersey has done witii its 
"General Intellectual Skills" (CIS) examination (which covers such areas as 
"gatiiering, analyzing, and presenting information"), or as several states have done 
widi respect to basic skills? If so, should goaJs be couched as minimum standards 
that all students should demonstrably meet, as "central tendencies" for college 
graduater as a body, or a: aspirational challenges to guide future development? 
This is a central policy dilemma in many states: "mandating adequacy" through 
mitiimum standards can potentially narrow cuniculum and leave more selective 
Institutions unaffected, as has happened for die most part in Florida; "challenging 
excellence" through higher standards on the otiier hand, can raise issues of equity 
tiiat often entail substantial political costs. 

A second issue is which goals, if any, should be held in common . Is it appropriate 
for state policy to articulate for all institutions die actual kinds of knowledge and 
skills required as, for example, is suggested by Virginia's "Curriculum for the 21st 
Century?" Or oueht state policy to be confmed to requiring institutions to address 
goals in common ai'?as - for example general education, student satisfaction, or 
major field achievement? Those states, such as Colorado, which have followed tljc 
latter course have found remarkable overlap in the actual content of institutional 
goal statements, once developed, ihough each was developed "indepcndentiy." 
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Rather than mandating uniform goal statements, this finding suggests a strategy of 
allowing them to emerge gradually. 

A final issue concerns who should be involved in the goal-setting process. 
Structuring a statewide dialogue about desirable outcomes, for most states, remains 
largely unknown territory. One alternative is to attack the problem by involving 
business and industry leaders direcdy with academics in the goal-set'ing process, as 
was practiced by the College Outcomes Evaluation Program (COEP) in New Jersey. 
Anotiier, as in Colorado, is to require institutions to provide evidence that their own 
assessment activities include efforts to determine die needs and perceptions of the 
various "communities" Uiat tiiey are intended to serve. 

2. What assessment information is useful for making sure that the cumculum provides 
die necessary skills and content ? 

In order to be useful for guiding improvement, information derived from assessment 
must meet several conditions. First, it must address multiple dimensions of student 
capacity and performance; capacities such as "critical thinking," for example, are 
notoriously resistant to useful general measures. Secondly, it must be provided in a 
form that is sufficientiy disaggregated to guide intervention; overall summaries of 
institutional performance will often mask substantial and important variations across 
departments and programs, or among different types of students. Finally, it must 
address instructional processes as well as obtained outcomes; if institutions know 
notiung about the actual delivery of instruction in die classroom, or about the 
choices tiiat students are making about what classes to take, it is next to impossible 
to know what to "fix" if outcomes deficiencies are detected. As a result, the most 
useful assessment information for guiding curriculum renewal is often the most 
difficult to meaningfully aggregate and summarize. 

And as in the case of goals, much turns on what is considered "the necessary skills 
and content" If the resolution to diis question is collective, unifomi performance 
information such as that provided by basic skills examinations in Florida or New 
Jersey may provide an appropriate answer, both are being used to direct statewide 
attention to needed cnanges in curriculum ~ most notably in the teaching of 
mathematics where important changes and improvements in performance have in fact 
occurred. But if die resolution to Uiis question is local, "accountability" may rest 
largely upon a demonstration that obtained information has been effectively used to 
address curricular issues. In the latter case, state policies that encourage and 
recognize information ii|e rather than information reporting may be most needed. 

3. How can assessment support departments and disciplines in assuming collective 
responsibilitv for the outcomes of college curricuia ? 

A critical condition for curricular reform is for faculty in diverse departments and 
disciplines to recognize their responsibility for undergraduate education as a whole. 
Current organizational and reward structurcc in most colleges and universities are 
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strongly at odds with this condition. One virtue of assessment is that it raises the 
question of collective responsibility in concrete ways that are difficult to ignore. 

The most visible place where issues of "joint product" arise is in the assessment of 
general education, and it is no coincidence that assessment in this arena has proven 
the most difficult for institutions to actually implement A major policy dilemma 
here is illustrated by the common decision to require institutions to repon on the 
assessment of "general education" as a separate enterprise, when arguably the kinds 
of outcomes claimed for "general education" are coincident with the college 
experience as a whole. But by not calling explicit attention to general education, 
state assessment policies risk encouraging an institutional response that may ignore 
collective products altogether. 

Additional "collective responsibilities" cut across institutional boundaries. In many 
state university systems, as many as a third of the baccalaureate degrees granted in 
a given year are "joint products" of university and community college instruction. 
Yet curricular articulation between two-year and four-year institutions remains a 
significant challenge. Here a primary contribution of assessment policy can be to 
formulate the question of "articulation" on different grounds. Instead of matching 
individual courses content-for-content, the policy question becomes, "what particular 
areas of knowledge and skill are required of students as a prerequisite for upper- 
division work?" One accomplishment of G^AST in Rorida has been to address this 
question in rudimentary form. More sophisticated examples include attempts to 
define core transfer competencies among community colleges in Texas and 
Washington. 
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ASSESSMENT AND TEACHING 



If linkages between state policy and college curriculum are indirect, those between policy 
and teaching practice are looser still. Yet a signiHcant development in the practice of 
assessment in the last few years has been an increased emphasis on improved pedagogy. 
In contrast to large-scale culminating examinations, assessment practitioners are increasingly 
urging faculty to "embed" assessment in regular coursework and to undertake a systematic 
program of "classroom research" to determine what teaching strategics are working and 
how snidents are actually experiencing the classroom. 

Emerging campus experience suggests tiiat techniques such as these are significantiy more 
likely to yield faculty engagement and positive change; but they at first appear to be far 
from the influence of state policy. Narrowing this distance, we believe, requires attention 
to three distinct policy questions: 

1. How can assessment help to inform the teaching and leamine process within the 
classroom? 

Assessment processes such as "classroom research" are notoriously difficult to 
mandate. Like the familiar "course evaluation" processes required at many 
institutions, success depends upon the perceived utility of the process to individual 
instructors. Requiring course evaluation, research has shown, has in itself had 
remarkably little impact on classroom behavior. As with the curriculum, therefore, 
the greatest impact of state policy in this area may be symbolic: it signals clearly 
that faculty must take responsibility for student learning raUier than, as many 
believe, simply providing an arena for it to happen. 

State assessment policies may nevertheless profoundly influence classroom behavior 
in at least two ways. Fu^t, they may help to determine who is actually in a 
particular college classroom. Coupling basic skills assessment with initial student 
placement and advisement, as is done in Tennessee and New Jers<ry, seems reliably 
to increase a student's chances of success, though it may also increase the amount 
of time needed to attain a degree. But mandatory placement can also raise 
significant equity questions - particularly if test bias becomes an issue, or if it is 
diificult to show a significant relationship between test performance and later 
performance in the curriculum. 

State assessment policy can also focus greater institutional attention upon classroom 
process . Are students engaging in "active learning" or "group study," as advocated 
by such sources as the recentiy-issued Wingspread "Seven Principles of Good 
Practice in Undergraduate Education"? Are they engaged in learning on their own, 
in parallel with classroom instruction? Are they choosing courses in a manner that 
allows them to avoid key learning experiences, such as librar>' research, interaction 
with a computer, or a sustained piece of writing or malysis? Some state 
assessment approaches (most recently, liie "Q-7" quality indicators project of the 
Minnesota State University System) are now attempting to address such questions. 
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How can assessment be used in desienine state policies that support good colleee 
teaching ? 

Addressing this quesdon suggests an important policy linkage between assessment 
and faculty development Eariy experiments in statewide collegiate skills assessment 
such as the New Jersey CIS Examination, suggest that students may systematically 
lack contextual knowleidge (a solid grasp of physical geography, fo example) that 
their instructors may take for granted in instructional delivery. Students may also 
profoundly misunderstand what is really being sought in a typical inteUectual task 
(in the CIS, for instance, the term "essay" had to be explicidy defined before 
meaningful responses were obtained). Both kinds of insights can be particularly 
helpful in structuring faculty development 

Considerable experience has also shown that designing assessment procedures can 
itself be a significant faculty development activity. Most college and university 
teaching faculty have had little or no explicit training ,n how to write good 
examination questions or how to explicitly recognize and foster specific aspects of 
student growth and development To the extent that state policies encourage this 
kind of faculty engagement they arc also pursuing faculty development 

How can assessment be linked to the institutional reward structure to support good 
teaching ? 

A major intent of state policy on assessment is deceptively simple: to shift a 
greater proportion of scarce institutional attention toward undergraduate education. 
Permanent shifts of attention, however, are generally accomplished only in the 
context of an underlying shift in incentives. So long as faculty perceive that their 
own reward structures to be predominantiy research-based and discipline-oriented, 
there is littie possibility of widespread attention to good teaching. 

But altering faculty reward structures on the basis of an activity as controversial as 
assessment poses critical policy challenges. First in examining the incentive 
strucrare surrounding teaching, it is iinportant to distinguish individual pavoffs from 
collective investments . Coupling the results of assessment to individual faculty 
rewards eitiier in the promotion and tenure system or in the form of additional 
dollar incentives for "exemplary" performance - risks incurring faculty behaviors, 
such as "teaching to the test" that are directiy counterproductive to improve<l 
teaching. More importantiy, coupling assessment with individual rewards may foster 
a level of competition among faculty that is direcUy antiUietical to tiieir assuming 
collective responsibility for cuiricular outcomes. A second issue concerns the 
degree to which "assessment" is seen as an additional, fundable, faculty 
responsibility, distinct from instruction itself. Not paying for the incurred costs of 
assessment on tlie one hand, risks sending the signal that the activity is seen as of 
low priority. Negotiating a full-cost "rate scale" for faculty participation in 
assessment as numy collective bargaining units now advocate, risks stifling the kind 
of creativity needed for a meaningful program. 
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Both issues highlight the importance of taking a broad, systemic view of the reward 
structure for teaching. At bottom, institutional experience shows that faculty will 
become involved in assessment primarily because they feel that it enables them to 
do a better job as teachers, and because they can, in fact, take collective pride in 
doing so. To the tent that resources are involved, the key may be to clearly 
channel resources .jward addressing concrete instructional problems that faculty 
identify, rather than to provide individual dollar rewards. 
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ASSESSMENT AND RESOURCE NEEDS 



In contrast to cuiriculum and teaching practice, resource allocation has always been a 
premier domain of state higher education policy. As a result, when thinking about ways to 
harness assessment as a policy tool for improvement, legislators and state board members 
can be quick to seek concrete linkages to allocation. Varieties of "performance funding" 
- often proposed, but to date actually implemented only in Tennessee - generally lead the 
list of proposed innovations. But the realities of linking assessment results directiy to 
resource issues can in practice prove complex. At minimum, we believe, the following 
questions must be addressed: 

1. How can assessment be related to the determination of resource needs ? 

To address this question coherentiy, "resource needs" must first be distinguished for 
institutions and for the system as a whole . In the latter, assessment results are 
potentially powerful tools for making an effective case about overall need. In both 
Florida and New Jersey, for example, lower- than-expected basic skills results have 
bolstered die case for greater invesmient in remedial and basic skiUs instruction, 
particularly among community colleges. But demonstrating need on the basis of 
visible shortfalls in performance can prove a risky long-term strategy, particularly if 
the reasons for shortfall are persistent and systemic. Indeed, higher education 
leaders have been historically sensitive about sharing "bad news" explicitly with 
state policy makers, even if it can potentially be used to make a case for increased 
investment 

At the institutional level, the use of assessment information to make the case for 
need encounters a similar dilemma. But here, state policy can in principle be more 
consistent Funding preference can be given, for example, to institutional special 
budget requests that make visible and effective use of locally-collected assessment 
information as has been the case, for example, in New Jersey's "Governor's 
Challenge Grant" or in tiie Colorado "Centers of Excellence" program. The key to 
success in such programs is tiiat they foster a healtiiy competition among institutions 
while at the same time linking new investments clearly to desired state purposes. 

2. How can assessment help to inform the allocation of resources ? 

The nature of the link between assessment and resource all cation is probably the 
most important single assessment policy question at the state level. If there is no 
link, assessment cannot in the long run be taken seriously by faculty and 
institutional administrators. But if die link is inconsistent, unclear, or even too 
directive, the chances of meaningful impact on teaching and learning can be equally 
reduced. 

Most states, for better or worse, have chosen not to link assessment results directly 
to resources. Partly, this is a tactical decision, based on the need to initiate what is 
known will be controversial process in as non-threatening a manner as possible. 
Where such linkages are present, they tend to be botii indirect and tied to the 
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process rather than the results of assessment (Both Virginia and New Jersey, for 
e^iunple, have invested heavily in marginal allocations to support assessment at the 
utstitutional level, as did Florida when CLAST was implemented several years ago.) 
>^1ien pressed, however, state leaders say they hope to make more visible use of 
assessment results through such processes as prognun review and improvement, 
faculty development, and s Jditional marginal incentive grant programs. 

Effectively linking assessment results to institutional resource allocation involves 
resolution of at least three major policy dilemmas. First, should the philosophy of 
allocation be driven by rewarding performance or by addressing need? The classic 
model of "performance funding" ties positive assessment results to increased 
allocation. Because it appears to provide incentives for improvement, performance 
funding has inherent appeal to legislators and state boards. In practice, however, it 
may direct investment away from where it is most needed, and may cause 
institutions to give undue attention to narrow conceptions of performance. Directing 
resources toward detected problems, on the otiier hand, can tend to obscure real 
excellence and innovation. Both situations can be exacerbated when, as in 
Colorado, resource consequences are negative in the form of a budget penalty. 

A second dilemma involves the level of resources at issue. For good reasons, most 
states continue to allocate resources to institutions on the basis of imputed cost - 
either tiirough formula or through incremental adjustments to base. What is 
generally at stake, therefore, are "marginal" investments of from 2% to 10% of 
available resources, allocated by a special process. But the higher the stakes, the 
greater the dilemma. If litUe is seen to be at stake, as in states like Missouri or 
Illinois, institutions may pay littie attention. If much is at stake, as in Tennessee, 
Colorado, or Virginia, the incentive may be to take few risks, to report only good 
news, and to see assessment primarily as an administrator's responsibility. In 
Tennessee, for example, one unintended consequence of performance funding is an 
institutional perception that assessment is "too important to be left to the faculty." 

A final dilemma concerns the actual mechanism to be used to allocate funds. On 
the one hand, equity demands the consistent application of clear criteria, lest 
institutions rightiy charge that assessment- based resource allocations are made 
capriciously. But, as in Tennessee, this may drive policy inexorably toward narrow 
quantitative criteria that can be "unambiguously" applied. The result may be a 
system littie different in function from a traditional enrollment-driven formula. The 
increasing use of "peer institutions" in a wide range of state budgetary mechanisms 
represents one attempt to get around this problem. Lack of clarity in the allocation 
process, on the other hand, can render it sufficienUy political that the real ends of 
improvement become obscure. 

How can assessment be used to examine and increase the "return on investment" of 
public resources in higher education ? 

An undoubted stimulus for assessment is a growing demand for evidence of "return 
on investment." Legislators and state board members are increasingly impatient 
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with claims that ths complexity of higher education renders infeasible any attempt to 
gather evidence of its effectiveness. As a result, many have been quick to press for 
assessment "solutions" that attempt to provide such evidence promptly and simply. 
Many in the academy, in tum, have been quick to condemn such motives as 
inconsistent with the real goal of assessment: improving teaching and learning. 

Before immediately characterizing these motives as antithetical, it is important to 
recognize Uie reality of the forces that lie behind them. One is a real decline in 
public cred ' bilitv for higher education as an enterprise; as costs have grown so has 
the public's conviction that colleges and universities are not a.<s focused on their own 
and society's needs as they once were. At the same time, political champions of 
higher education increasingly believe tiiat thiey need better evidence to "sell" needed 
increases to their doubting colleagues. More and more, they speak of such evidence 
in corporate language, calling for the equivalent of "shareholders* reports" or 
"annual earnings statements." Ironically, when pressed, their motives resemble those 
of the academy: they want assessment processes that are real, engaging, complex, 
and that have the potential to actually have an impact on undergraiduate classrooms. 
But they must also meet the demands of a constituency that wants to keep things 
straightforward, and that is growing impatient about what is seen as 
unresponsiveness. 

Significant gaps between the language and timeframes of the academic and political 
worlds raise aidditional policy issues. If "improvement" is indeed a long-term 
enterprise, what can the public be told in the short run ? Once it is known that 
"assessment" is occurring, as in New Jersey, public pressures for early disclosure 
become enormous. Should preliminary results be disclosed, no matter how 
uncertain? Or, as in South Dakota, should institutions be allowed to protect the 
conHdentiality of their individual results for a limited period while remedial actions 
are taken? Or should the emphasis of reporting be placed upon communicating 
institutional actions taken in response to assessment, as in Virginia or Colorado? 
And if so, is it necessary to report results as well as remedies, if only to 
demonstrate that a credible process is in fact in place? Finally, who are the 
primary constituencies for reporting? Should they be legislators and public officials 
primarily, who can at least minimally be expected to know the policy contexts 
within which assessment information belongs, or should reporting be extended to 
prospective students and employers of graduates, as has been posed in California, 
and as is currentiy called for in new "track rricord disclosure" rales for federally- 
supported occupational and technical programi/* 

At bottom, of course, evidence of "return on investment" is provided by 
improvement itself. State policies which can in fact foster improvement in the long 
run, will at the same time achieve "accountability" as a by-product. 
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